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We present the results of a search for the flavor changing neutral current decay B® — > fi + yT using 
6.1 fb _1 of pp collisions at \fs = 1.96 TeV collected by the DO experiment at the Fermilab Tevatron 
Collider. The observed number of candidates is consistent with background expectations. The 
resulting upper limit on the branching fraction is B(B° — ► < 5.1 x 10~ 8 at the 95% C.L. 

This limit is a factor of 2.4 better than that of the previous DO analysis and the best limit to date. 

PACS numbers: 12.15.Mn, 13.20.He, 14.40.Nd 



The standard model (SM) provides an accurate de- 
scription of current observations in high energy physics 
experiments, in particular precision electroweak measure- 
ments and flavor physics observables. A flavor changing 
neutral current (FCNC) process is an apparent transi- 
tion between quarks of different flavor but equal charge. 
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In the SM, the FCNC processes are forbidden at first 
order. They can occur at second order only through 
Glashow-Iliopoulos-Maiani (GIM) jl[ suppressed box and 
penguin diagrams. The decay B® —> [i + fi~ Q is an 
example of such a process, as shown in Fig. [TJ Un- 
like other FCNC decays this decay rate is further sup- 
pressed by helicity factors in the final state |3j. 
The SM expectation for the branching fraction of this 
decay is (3.6 ± 0.3) x 10" 9 0]. The decay amplitude for 
— > /i + pT can be enhanced by several orders of magni- 
tude in some extensions of the SM. For example, in some 
supersymmctric models such as the minimal supersym- 
metric standard model this decay can occur through the 
mediation of superpartners of the SM intermediate par- 
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FIG. 1: Examples of Feynman diagrams for FCNC decays: 
(a) box diagram, (b) penguin diagram. 



tides as well as particles from the extended Higgs sector. 
This extended contribution becomes larger if the value 
of tan/3, the ratio of the vacuum expectation values of 
the two neutral Higgs fields, is large [El-Hoj]. Similarly, 
in some supersymmetric models with i?-parity violating 
couplings this enhancement can be present even in 
the regime of low tan /3. Improved limits on the branch- 
ing fraction of this decay can be used to set limits on 
the parameter space of supersymmetric models and other 
new theories. Since the predicted rate for this process 
in the SM is beyond the current experimental sensitiv- 
ity at the Tevatron, the observation of this decay would 
necessarily imply physics beyond the SM. Similar anni- 
hilation topologies have also been studied for B° and B + 
decays [12rll6l ]. 

In this Letter, we report on a search for the rare decay 
Bg — > \i + pT using 6.1 fb -1 of integrated luminosity col- 
lected by the DO detector. Presently, the best experimen- 
tal bound for the branching fraction of B(B° S — » < 
5.8 x 1(T 8 at the 95% C.L. is given by the CDF Collabo- 
ration fl6l | . Our previous result for this search was based 
on 1.3 fb -1 of integrated luminosity and set a bound for 
the branching fraction B(B® —> < 1-2 x 10~ 7 at 

the 95% C.L. 0. 

The DO detector [l8| has a central tracking system, 
consisting of a silicon microstrip tracker (SMT) [19( and a 
central fiber tracker (CFT), both located within a 2 T su- 
perconducting solenoidal magnet, with designs optimized 
for tracking and vertexing at pseudorapidities 77 1 < 3 and 
\rj\ < 2.5, respectively, where r\ — — In [tan (8/2)], and 9 
is the polar angle with respect to the proton beam direc- 
tion. An outer muon system, covering < 2, consists 
of a layer of tracking detectors and scintillation trigger 



counters in front of 1.8 T toroids, followed by two similar 
layers after the toroids [20}. The trigger and data ac- 
quisition systems are designed to accommodate the high 
instantaneous luminosity of the Tevatron Run II that 
started in 2001. In summer 2006, the SMT detector was 
upgraded by inserting an additional layer of silicon mi- 
crostrip detectors, Layer [21], close to the beampipe. 
The data-taking period before the Layer installation 
is referred to as Run Ha, and the period afterwords is 
referred to as Run lib. The two data sets are analyzed 
separately. 

All data collected up to June 2009 are included in this 
analysis. The integrated luminosities for the Run Ha 
and Run lib data sets are 1.3 fb -1 and 4.8 fb -1 , respec- 
tively. Events arc recorded using a set of single muon 
triggers, dimuon triggers, and triggers that select pp in- 
teractions based on energy depositions in the calorimeter. 
B® — > /i + /i~candidates are formed from pairs of oppo- 
sitely charged muons identified by extrapolating tracks 
reconstructed in the central tracking detectors to the 
muon detectors, and matching them with information 
from the muon system. The muon selection has been 
updated with respect to the previous analysis [13] , yield- 
ing 10% higher acceptance while keeping the fraction of 
misidentified muons below 0.5%. Each muon is required 
to have a transverse momentum pt^ > 1.5 GeV, and to 
have hits in at least two layers of both the CFT and 
the SMT. The B® candidate is required to have a recon- 
structed three-dimensional (3D) decay vertex displaced 
from the interaction point with a transverse decay length 
significance Lt/<Jl t > 3 to reduce prompt dimuon back- 
ground, where Lt — It • Pt/\Pt\- The vectors It and pj, 
are, respectively, the vector from the interaction point 
to the decay point and the transverse momentum vector 
of the B® meson in the transverse plane. The pp inter- 
action vertex is found for each event using a beam-spot 
constrained fit as described in 
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Events are selected 
if the reconstructed invariant dimuon mass, to w , is be- 
tween 4.0 GeV and 7.0 GeV. 

To further suppress the background we use the follow- 
ing discriminating variables: the transverse momentum 
of the Bg candidate p|?, the pointing angle, Lt/ctl t , the 
decay vertex fit x 2 > the smaller impact parameter sig- 
nificance (S/as) of the two muons, imii(5/as), and the 
smaller pt^ of the two muons, min(p^). The pointing an- 
gle is defined to be the 3D opening angle between the 
B® meson momentum vector and the displacement vec- 
tor from the interaction to the dimuon vertex. The im- 
pact parameter S is defined to be the distance of clos- 
est approach of the track to the interaction point in 
the transverse plane, and as is its uncertainty. We use 
a Bayesian Neural Network (BNN) [2" 
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ate classifier with the above variables to distinguish sig- 
nal events from background. The BNN is trained us- 
ing background events sampled from the sideband re- 
gions (4.5 GeV < < 5.0 GeV and 5.8 GeV < 
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FIG. 2: Distributions of j3: (a) — > fi + fi signal and side- 
band events, (b) B + — > J/ip(fi + fi~)K + data and simulation. 



TO W < 6.5 GeV) and simulated signal events. To sim- 
ulate the B® signal, we generate Monte Carlo events 
using the PYTHIA [25[ event generator, interfaced with 
the evtgen (2|| decay package. We simulate the detec- 
tor response using GEANT [g?]]. Multiple interactions are 
modeled by overlaying randomly triggered data events 
on top of the simulated hits in the detector. The distri- 
butions of the BNN output /3 for the B° signal and the 
sideband events as well as the B + — ► J/ip(fj, + fj,~)K + 
control sample are shown in Fig. [3J We define the 
— > signal region to be 0.9 < /3 < 1.0 and 

5.0 GeV < m MM < 5.8 GeV where there is a clear sep- 
aration between signal and background. This region is 
determined by optimizing the expected sensitivity of the 
search. We prepare two-dimensional (2D) histograms of 
m-fifi vs. /3 dividing the signal region into several bins to 
improve the sensitivity relative to using a single bin. 

The dominant source of background dimuon events is 
from decays of heavy flavor hadrons in bb or cc produc- 
tion. To study this background contribution, we gener- 
ate inclusive dimuon Monte Carlo samples with PYTHIA 
generic QCD processes that include all bb or cc produc- 
tion processes. The dimuon background events can be 
categorized by two types: (i) B(D) — > n + vX.B(D) — > 
ji~uX' double semileptonic decays where the two muons 
originate from different 6(c) quarks, yielding dimuon 
masses distributed over the entire signal region, and 
(ii) B — > fi^vD^D — » [i~DX sequential semileptonic 
decays, resulting in m MM predominantly below the B 
hadron mass. The simulated dimuon mass distributions 
for both background sources after requiring /3 > 0.8 are 



parametrized using an exponential function to estimate 
the number of background events in the signal region 
after fitting the dimuon mass in the data sideband re- 
gions, 4.0 GeV < m w < 5.0 GeV and 6.0 GeV < 
"V^ < 7.0 GeV, in each /? bin. The uncertainty on 
this background estimate is dominated by the statistical 
uncertainty of the sideband sample (10-35%). In ad- 
dition, we consider background contributions from B° 
and B® decays B — » h + h'~, where h + and h'~ represent 
a charged kaon or pion. The muon identification effi- 
ciency and the fractions of pions and kaons misidentificd 
as muons are evaluated using samples of J/tp — > 
and D° -> K+ir' in B -)• iivD° decays. B° -> K+R- 
decay is the largest contribution in the B — > h + h'~ back- 
grounds and that is expected to be 0.13 ±0.10 events for 
Run Ha and 0.36 ± 0.27 events for Run lib in the signal 
region, where the uncertainty is dominated by the sta- 
tistical uncertainty on the fraction of misidentification. 
The B —7- h + h'~ background contribution is thus found 
to be negligible (see below). 

The branching fraction B(B° S — > /i + /i~) is computed by 
normalizing the number of events, N(B®), to the number 
of reconstructed B + — > J/tjj(n + fi~)K + events, N(B + ): 

where the parameters and e^o are the reconstruction 
efficiencies for B + — >• J/ip(fi + /i~)K + and B® — > 
respectively. They are estimated from simulations. We 
use B(B+) = B{B+ ->• J/i/jK+) x B {J/ijj -> /i+M") = 
(5.97 ± 0.22) x 10" 5 [li] and the ratio of 5-hadron pro- 
duction fractions f u /f s = 3.86±0.59 [2!|. The simulated 
mass resolution of the DO detector for the — > /i + fi~ 
is ~ 120 MeV and is therefore insufficient to readily 
separate from B° leptonic decays. In this anal- 
ysis, we assume that there are no contributions from 
B° — > decays, since this decay is suppressed by 

\v td /v ts \ 2 «o.o4 BiHU. 

A sample of B + — > J/ip(fi + /i~)K + events is selected 
using all but the (3 selection requirements, with an ad- 
ditional requirement of plf > 1 GeV for the kaon can- 
didate. By performing a binned likelihood fit with the 
J/ipK + invariant mass distribution in data, we observe 
N(B+) = 14340±665 events for Run Ha and 32463±875 
events for Run lib, where the uncertainty is only statis- 
tical. The statistical significance of the B + signal yield 
in Run lib is higher than that in Run 11a although the 
lower yield per the integrated luminosity. The J/i/jK + 
invariant mass distribution is shown in Fig. [3] A system- 
atic uncertainty of 2% on the B + yield is found by vary- 
ing the fit parametrization. The efficiency for the addi- 
tional kaon track in B + — > J/if}(fi + ^)K + decays is cal- 
ibrated using the ratio of B° J/?Hm + ^)K*°(K + n-) 
to B + — > J/i/j(/j, + /j,~)K + data with an uncertainty of 
7.5%. The trigger efficiency depends on the muon trans- 
verse momentum pi^. This is modeled by comparing the 
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FIG. 3: The J/ipK + invariant mass distribution of B + candi- 
dates. The dashed line represents the B + signal distribution 
obtained from the fit (solid line). 



Pt 

trol sample requiring a pj, independent trigger and then 
applying the ratio to the simulated events as a p!^ depen- 
dent weight factor. A possible dependence of this weight 
factor on the dimuon kinematics is evaluated by choos- 
ing another sample at higher dimuon masses; this effect 
is found to be less than 1%. The p^ spectra in the B° 
and B + simulations are corrected following comparisons 
of the B + — > J/ip(p + p~)K + in data and simulation. A 
similar correction is obtained from B° s — > J/ip(f) decays, 
and the difference between the two is assigned as an un- 
certainty of 6.5%. The product of the factors multiplying 
N(B®) on the right-hand side of Eq. [I] is called the sin- 
gle event sensitivity. We find a single event sensitivity 
(4.9 ± 1.0) x ICT 9 for Run Ila and (1.84 ± 0.36) x lfr 9 
for Run Jib in the signal region. Using the SM prediction 
of B{B° S -> fi+fi-) 4], there are 0.74 ± 0.17 events in 
Run Ila and 1.95 ±0.42 events in Run lib expected in the 
signal region. Aside from the background uncertainty, 
the largest uncertainty of 15% common to Run Ila and 
Run lib comes from the fragmentation ratio, f u / f s . 

We compute the final sensitivity using 2D histograms 



of 



vs. p of the signal and the backgrounds by 



combining the sensitivity of each bin taking into ac- 
count the correlated uncertainties. In addition to the 
uncertainty on the signal normalization, we add uncer- 
tainties on the expected mass and its resolution in 
the calculation. Additional uncertainties on the dimuon 
background distributions are assigned to allow for pos- 
sible variation in the background m w distribution as 
a function of j3. The resulting median expected lim- 
its are B(B° S -> /i+^O < 8.5 x 10~ 8 (6.8 x 10~ 8 ) for 
Run Ila, and 4.6 x 10~ 8 (3.7 x 10~ 8 ) for Run Kb at the 
95% (90%) C.L. and the combined median expected limit 
is B{B° S -)■ a« + aO < 4.0 x 10- 8 (3.2 x lO" 8 ). The lim- 
its are calculated from Eq. [T] using the semi-Frequentist 
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FIG. 4: The distribution of m M(J in the highest sensitivity 
/3 region (a), and the distribution of /3 in the highest sen- 
sitivity m,j M region (b) for data (dots with uncertainties), 
expected background distribution (solid line), and the SM 
signal distribution multiplied by a factor of 100 (dotted- 
dashed line). The dimuon background contributions from the 
B(D) — > fj, + uX, B(D) —¥ yTvX' decays (dashed line) and the 
B — > n + vD, D — > ^TvX decays (dotted line) are also shown. 



confidence level approach (CL S ) [32j-|34j with a Poisson 
log-likelihood ratio test statistic. The limit incorporates 
Gaussian uncertainties on the signal efficiency and the 
background. This expected limit is a factor of 2.4 better 
than the expected limit of 9.7 x 10~ 8 at the 95% C.L. of 
the previous DO result [l7|, where 10% of this improve- 
ment results from changes in the analysis technique. 

After finalizing the selection criteria and all system- 
atic uncertainties, we study events in the signal region. 
There are 256 events for Run Ila, and 823 events for 
Run lib observed in the signal region where the ex- 
pected number of background events is 264 ± 13 events 
for Run Ha and 827 ± 23 events for Run lib. The 
observed distributions of dimuon events in the high- 
est sensitivity region are shown in Fig. 01 The ob- 
served number of events is consistent with the back- 
ground expectations. We extract 95% (90%) C.L. limits 
of B(B° S ->• fi+fi-) < 8.2 x 10~ 8 (6.5 x 10~ 8 ) for Run Ha 
and 6.5 x 10~ 8 (5.3 x 10~ 8 ) for Run lib. The resulting com- 
bined limit is B(B° -» n+p-) < 5.1 x 10~ 8 (4.2 x 10~ 8 ) 
at the 95% (90%) C.L. The probability for the expected 
background distributions to fluctuate to the observed 
data distributions is 31%. 

In conclusion, we have reported a search for the rare 
decay B® — > p + fi~ using 6.1 fb^ 1 of pp collisions collected 
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by the DO experiment at Fermilab Tevatron Collider. We 
observe no evidence for physics beyond the SM and set a 
limit of B(B° -> < 5.1 x 10~ 8 (4.2 x 1(T 8 ) at the 

95% (90%) C.L. This result is more stringent than the 
previous results 16, [13] and the best limit to date. 
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